We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning. VILLA consists of two training stages: (i) task-agnostic adversarial pre-training; followed by (ii) task-specific adversarial finetuning. Instead of adding adversarial perturbations on image pixels and textual tokens, we propose to perform adversarial training in the embedding space of each modality. To enable large-scale training, we adopt the ``free'' adversarial training strategy, and combine it with KL-divergence-based regularization to promote higher invariance in the embedding space. We apply VILLA to current best-performing V+L models, and achieve new state of the art on a wide range of tasks, including Visual Question Answering, Visual Commonsense Reasoning, Image-Text Retrieval, Referring Expression Comprehension, Visual Entailment, and NLVR2.

Weaknesses: Besides the strength of the paper, I have some concerns about the paper. In this paper, the authors show that by adding adversarial perturbations into the embedding, the model can improve the performance on final downstream tasks. This is great, however, the paper didn't answer whether the proposed method can perform better in the adversarial attack? What is the connection between adding noise in embedding space and pixel/token space? There are multiple ways to test how the proposed method is more robust, for example: - Some downstream tasks focus on paraphrasing, there is a vqa-rephrasing dataset, and I am curious whether injecting the adversarial noise into the embedding space will lead to better performance on this dataset?

All four reviewers support acceptance for the contributions, notably the idea of using adversarial perturbations for training transformer-based vision-language models and successfully demonstrating this idea experimentally on a 6 standard vision&language tasks / datasets leading to SOTA results, all in a clearly written and organized paper. I agree to these observations and also recommend acceptance of this strong paper. The concerns the reviewers had, have been successfully addressed in the author response and I expect the authors will follow through with their promise to release all code and pre-trained models and revise the paper with correction, clarifications and additions from the rebuttal, including results on VILLA_LARGE and maybe add more additional qualitative examples in the appendix.

We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V L) representation learning. VILLA consists of two training stages: (i) task-agnostic adversarial pre-training; followed by (ii) task-specific adversarial finetuning. Instead of adding adversarial perturbations on image pixels and textual tokens, we propose to perform adversarial training in the embedding space of each modality. To enable large-scale training, we adopt the free'' adversarial training strategy, and combine it with KL-divergence-based regularization to promote higher invariance in the embedding space. We apply VILLA to current best-performing V L models, and achieve new state of the art on a wide range of tasks, including Visual Question Answering, Visual Commonsense Reasoning, Image-Text Retrieval, Referring Expression Comprehension, Visual Entailment, and NLVR2.